System for 3d surveying by a ugv and a uav with automatic provision of referencing of ugv lidar data and uav lidar data

ABSTRACT

A system for 3D surveying of an environment by an unmanned ground vehicle (UGV) and an unmanned aerial vehicle (UAV) has two lidar devices. A reference unit has a first and a second marker in a spatially fixed arrangement. An automatic detection of the first marker is carried out for a coordinative measurement by the first lidar device to determine relative position data for providing relative position information of the first marker with respect to the first lidar device. The relative position data and spatial 3D information is used for an automatic detection and a coordinative measurement of the second marker by the second lidar device. The coordinative measurements are used for a referencing of lidar data of the UGV lidar device and lidar data of the UAV lidar device with respect to a common coordinate system.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application claims priority to European Patent ApplicationNo. 21193138.1 filed Aug. 25, 2021, which application is incorporatedherein by specific reference in its entirety.

BACKGROUND Field

The present invention relates to a system for providing 3D surveying ofan environment by an unmanned ground vehicle (UGV) and an unmannedaerial vehicle (UAV).

Description of Related Art

By way of example, three-dimensional surveying is used to assess anactual condition of an area of interest, e.g. a restricted or dangerousarea such as a construction site, an industrial plant, a businesscomplex, or a cave. The outcome of the 3D surveying may be used toefficiently plan next work steps or appropriate actions to react on adetermined actual condition.

Decision making and planning of work steps is further aided by means ofa dedicated digital visualization of the actual state, e.g. in the formof a point cloud or a vector file model, or by means of an augmentedreality functionality making use of the 3D surveying data.

3D surveying often involves optically scanning and measuring anenvironment by means of a laser scanner, which emits a laser measurementbeam, e.g. using pulsed electromagnetic radiation. By receiving an echofrom a backscattering surface point of the environment a distance to thesurface point is derived and associated with an angular emissiondirection of the associated laser measurement beam. This way, athree-dimensional point cloud is generated. For example, the distancemeasurement may be based on the time of flight, the shape, and/or thephase of the pulse.

For additional information, the laser scanner data may be combined withcamera data, in particular to provide high-resolution spectralinformation, e.g. by means of an RGB camera or an infrared camera.

However, acquiring the 3D data can be cumbersome and in some cases evendangerous for a human worker. Often, access to a specific area isprohibited or severely restricted for a human worker.

Nowadays, robotic vehicles, particularly autonomous robotic vehicles,are increasingly used to facilitate data acquisition and to reduce riskson human workers. 3D surveying devices used in combination with suchrobotic vehicles are typically configured to provide surveying dataduring movement of the robotic vehicle, wherein referencing data provideinformation on a trajectory of a data acquisition unit, e.g. positionand/or pose data, such that surveying data acquired from differentpositions of the data acquisition unit can be combined into a commoncoordinate system.

The 3D surveying data may then be analyzed by means of a featurerecognition algorithm for automatically recognizing semantic and/orgeometric features captured by the surveying data, e.g. by means ofusing shape information provided by virtual object data from a CADmodel. Such feature recognition, particularly for recognizing geometricprimitives, are nowadays widely used to analyze 3D data.

Many different types of autonomous robotic vehicles are known. Forexample, ground based robotic vehicles may have a plurality of wheelsfor propelling the robot, typically having sophisticated suspension tocope with different kinds of terrain. Another widely used type is alegged robot, e.g. a four-legged robot, which is often able to handletough terrain and steep inclines. Aerial robotic vehicles, e.g.quadcopter drones, allow further versatility to survey areas that aredifficult to access, but often to the expense of less surveying timeand/or sensor complexity due to limited often load capacity and batterypower.

Unmanned Arial Vehicles (UAV) and Unmanned Ground Vehicles (UGV) are forthemselves state-of-the-art platforms for multilateral use. Equippedwith imaging and lidar sensors, these platforms provide for autonomouspath planning and for autonomously moving an acquisition unit foracquiring 3D surveying and reality capture data.

For movement control and path planning, the autonomous robotic vehicleis often configured to autonomously create a 3D map of a newenvironment, e.g. by means of a simultaneous localization and mapping(SLAM) functionality, using data from sensors of the robotic vehicle.

In the prior art, movement control and path planning for the surveyingcampaign are predominantly governed by making use of inbuilt visualperception sensors of the autonomous robot. Acquisition and use of 3Dsurveying data are typically decoupled from acquisition and use ofcontrol data to move the robot.

In prior art robotic vehicles, often a tradeoff has to be made betweenfield-of-view and viewing distance on the one side and reactivity (e.g.for obstacle detection and initiating an evasive maneuver) on the otherside, which limits movement speed of the robot. Often, the robot only“sees” its immediate surroundings, which provides efficient reactivityto cope with obstacles and terrain changes, while larger scale pathcontrol is provided by predefined environment models and guidinginstructions. For example, this limits applicability of mobile 3Dsurveying by autonomous robotic vehicles in unknown terrain. In knownterrain, predefining paths to be followed is cumbersome and ofteninvolves skilled personnel to take into account various measurementrequirements such as a desired point density, measurement speed, ormeasurement accuracy.

The combination of multiple autonomous robotic vehicles providesflexibility in surveying large and varied areas (e.g. different soilconditions, measurements from the ground and from the air, etc.). Eachmobile surveying device may provide 3D surveying data. By way ofexample, from each measurement location a surveying device generates aso-called local 3D point cloud providing multiple measurement pointsreferenced to a common coordinate system relative to the surveyingdevice. When moving the surveying device, the local point cloudsdetermined in different locations of the surveying device have to berelated to each other by a process called referencing, point cloudregistration, point set registration, or scan matching in order to forma so-called 3D survey point cloud of the respective surveying device. Inaddition, the 3D survey point clouds from different surveying deviceslocated on different autonomous robotic vehicles have to be referencedto each other in order to form a so-called “combined” 3D survey pointcloud.

For referencing local point clouds of the same surveying device, oftenadditional information such as data from an inertial measurement unitand a simultaneous localization and mapping (SLAM) unit provided by thesurveying device are used.

Often, referencing of different survey point clouds from differentsurveying devices is cumbersome and only possible in post-processing.For example, the referencing process is made even more difficult by thefact that often different kinds of surveying sensors are used, e.g.wherein the sensors of different surveying devices provide differentpoint densities, field-of-views, and distance resolution. In case imagedata are used, different image distortions have to be accounted for,etc.

In order to save computing time, point matching often involves anoperator who manually identifies and links matching features indifferent survey point clouds of different surveying devices. Having anessentially real-time fusion of different 3D data of different devicesis thus often not possible or error prone and susceptible tointerruption.

SUMMARY

In some embodiments, the present invention provides an improved systemfor mobile 3D surveying, which has increased applicability, particularlyin view of surveying varying and extensive terrain.

In some embodiments, the present invention provides a mobile 3Dsurveying system, which is easier to handle and is more robust againstinterruption.

In some embodiments, the invention can be achieved by realizing at leastpart of the features of the independent claims. Features which furtherdevelop the invention in an alternative or advantageous manner aredescribed in the dependent patent claims.

In some embodiments, the invention relates to a system for providing 3Dsurveying of an environment, wherein the system comprises a first and asecond lidar device. One of the first and the second lidar device, inthe following referred to as UGV lidar device, is specifically foreseenfor a montage on an unmanned ground vehicle and configured to generateUGV lidar data to provide a coordinative scan of the environmentrelative to the UGV lidar device. The other of the first and the secondlidar device, in the following referred to as UAV lidar device, isspecifically foreseen for a montage on an unmanned aerial vehicle andconfigured to generate UAV lidar data to provide a coordinative scan ofthe environment relative to the UAV lidar device. The system isconfigured to provide a referencing of the UGV lidar data and the UAVlidar data with respect to a common coordinate system for determining a(combined) 3D survey point cloud of the environment.

In some embodiments, the system is configured to provide a localizationof the UGV lidar device in a digital 3D model based on the UAV lidardata and, vice versa, to provide a localization of the UAV lidar devicein a digital 3D model based on the UGV lidar data.

In some embodiments, the system comprises a reference unit comprising afirst and a second marker, wherein the first and the second marker arein a spatially fixed arrangement with respect to each other and each ofthe first and the second marker is configured as target for acoordinative measurement of the respective marker by a lidar device. Thesystem is configured to carry out an automatic detection of the firstmarker and to carry out a coordinative measurement of the first markerby the first lidar device to determine relative position data providingrelative position information of the first marker with respect to thefirst lidar device. Furthermore, the system is configured to take intoaccount the relative position data and a spatial 3D information on thespatially fixed arrangement of the first and the second marker withrespect to each other to carry out an automatic detection of the secondmarker and to carry out a coordinative measurement of the second markerby the second lidar device. The coordinative measurement of the firstmarker and the coordinative measurement of the second marker are thentaken into account to provide the referencing of the UGV lidar data andthe UAV lidar data with respect to the common coordinate system.

By way of example, both the UGV lidar device and the UAV lidar deviceare provided with a combined start/end reference. The first and thesecond markers are referenced to each other by design, e.g.mechanically, wherein one marker allows to be well detected by thestarting/landing UAV. For example, in the nominal setup of the referenceunit, the first marker is arranged in the vertical plane and the secondmarker is arranged in the horizontal plane. The first marker then allowsto be well detected by an AGV passing by while the second marker allowsto be well detected by a starting/stopping UAV. The two markers thusprovide a base reference between UAV and UGV, allowing to spatially fusethe UGV lidar data and the UAV lidar data.

Accordingly, in some embodiments, one of the first marker and the secondmarker, in the following referred to as UGV marker, is configured thatin a nominal setup of the reference unit it is spatially arranged insuch a way that the UGV lidar device can carry out a coordinativemeasurement of the UGV marker, wherein the coordinative measurement ofthe UGV marker is carried out from a sideways looking field-of-viewassociated with the montage of the UGV lidar device on the UGV. Theother of the first and the second marker, in the following referred toas UAV marker, is configured that in the nominal setup of the referenceunit, it is spatially arranged in such a way that the UAV lidar devicecan carry out a coordinative measurement of the UAV marker, wherein thecoordinative measurement of the UAV marker is carried out from adownward looking field-of-view associated with the montage of the UAVlidar device on the UAV.

In some embodiments, the system is configured that the coordinativemeasurement of the first marker is carried out by the UGV lidar deviceand the coordinative measurement of the second marker is carried out bythe UAV lidar device, wherein the automatic detection of the secondmarker and the coordinative measurement of the second marker by the UAVlidar device is carried out at each take-off and landing of the unmannedaerial vehicle.

For example, the system is configured that the relative position data iscontinuously updated so that the relative position information providescontinuously updated spatial information about the arrangement betweenthe first marker and the UGV lidar device.

In some embodiments, a further marker on the UGV may be used to providea link between UAV and UGV along the mobile mapping process. Forexample, this provides a larger baseline to overcome inaccuracies of thereferenced start marker. Accordingly, in a further embodiment, thesystem comprises a further marker (in addition to the first and thesecond marker), which is specifically foreseen for a montage on theunmanned ground vehicle. The UAV lidar device is configured toautomatically carry out a coordinative measurement of the further markerand the system is configured to take into account the coordinativemeasurement of the further marker to provide the referencing of the UGVlidar data and the UAV lidar data with respect to the common coordinatesystem.

In some embodiments, spatial localization between UAV lidar device andUGV lidar device may be based on sparse maps from imagery and/or lidardata. For example, a sparse map is generated by a camera and/or lidardevice of either the UAV or UGV. The corresponding UGV or UAV thenlocalizes (in real-time) within the sparse map. Accordingly, in afurther embodiment, the system comprises a visual pick-up deviceconfigured to be arranged on the unmanned ground vehicle or the unmannedaerial vehicle, e.g. wherein the visual pick-up device is a camera orone of the UGV lidar device or the UAV lidar device. The system isconfigured to generate a sparse map using the visual pick-up device andto carry out a localization of the UGV lidar data or the UAV lidar datain the sparse map.

By way of example, the sparse map is generated by photogrammetrictriangulation, e.g. so-called structure from motion, and thelocalization comprises a first referencing between the UGV lidar dataand the UAV lidar data. Then, after the first referencing, a secondreferencing between the UGV lidar data and the UAV lidar data is carriedout based on point-cloud matching between the UGV lidar data and the UAVlidar data, wherein the sparse map is referenced with respect to a knowndigital model of the environment.

For example, known methods for the point-cloud matching includeiterative closest point-to-point, iterative closest point-to-plane,robust point matching, and Kernel correlation point set registration.The known digital model may be at least one of a digital buildinginformation model (BIM), a computer aided design model (CAD), and adigital model, e.g. a vector file model, generated from coordinativescan data provided by a terrestrial laser scanner (TLS), a mobilemapping system, or a photogrammetric capture device.

In some embodiments, the system is configured to access assignment data,which provide the spatial 3D information on the spatially fixedarrangement of the first and the second marker with respect to eachother.

Alternatively, or in addition, at least one of the first and the secondmarker comprises a visible code, e.g. a barcode or matrix barcode, whichprovides the spatial 3D information on the spatially fixed arrangementof the first and the second marker with respect to each other. Here, thesystem is configured to determine the spatial 3D information on thespatially fixed arrangement of the first and the second marker withrespect to each other by using a visual pick-up device, e.g. a camera ora barcode laser scanning device.

For example, the first and the second markers are embodied to bemechanically compatible with standard markers used in the prior art assurvey control points. They may be particularly embodied that they canbe measured with standard survey equipment such as a total station. Themarker may further be embodied that they can be automatically identifiedand detected in a point cloud software. For example, the markers containcoded information that can be read by the UGV and/or UAV lidar deviceand by a point cloud software during post processing, while at the sametime the markers have a visual feature that allows a measurement withtotal station.

In some embodiments, the first and the second marker are arranged on acommon component such that the relative spatial arrangement of the firstand the second marker is mechanically fixed.

By way of example, the common component further comprises an alignmentindicator, e.g. a bubble level, providing for a visual determination ofan alignment of the common component with respect to an outer coordinatesystem or with respect to a cardinal direction to establish the nominalsetup.

In some embodiments, at least one of the first and the second (or thefurther marker, see above) marker comprises a visually detectablepattern, e.g. provided by areas of different reflectivity, differentgray scales and/or different colors. The system is configured todetermine a 3D orientation of the pattern by determining geometricfeatures in an intensity image of the pattern, wherein the intensityimage of the pattern is acquired by a scanning of the pattern with alidar measurement beam of the UGV lidar device or the UAV lidar deviceand a detection of an intensity of a returning lidar measurement beam. Aplane fit algorithm is carried out in order to determine an orientationof a pattern plane, by analyzing an appearance of the geometric featuresin the intensity image of the pattern, and the system is configured totake into account the 3D orientation of the pattern for providing thereferencing of the UGV lidar data and the UAV lidar data with respect tothe common coordinate system.

By way of example, the pattern comprises a circular feature and thesystem is configured to identify an image of the circular feature withinthe intensity image of the pattern. The plane fit algorithm isconfigured to fit an ellipse to the image of the circular feature and,based thereof, to determine the orientation of the pattern plane. Forexample, the system is further configured to determine the center of theellipse and to derive aiming information for aiming with the lidarmeasurement beam to the center of the ellipse. The center of the ellipsemay then be used as aiming point to further determine a 3D position ofthe marker, e.g. allowing to determine and take into account a 6DoF pose(six degrees of freedom, position and orientation) of the marker.

In some embodiments, the pattern comprises inner geometric features,e.g. comprising rectangular features, which are enclosed by the circularfeature. For example, the inner geometric features provide informationon an alignment of the common coordinate system or an outer coordinatesystem, and/or the spatial 3D information on the spatially fixedarrangement of the UGV marker and the UAV marker with respect to eachother.

In some embodiments, the first and the second marker each comprise avisible indication of the directions of at least two, particularlythree, of the three main axes which span the common coordinate system,wherein the system is configured to determine the directions of thethree main axes by using the UGV lidar device and the UAV lidar device,and to take into account the directions of the three main axes forproviding the referencing of the UGV lidar data and the UAV lidar datawith respect to the common coordinate system.

In some embodiments, the coordinative scan of the environment by the UGVlidar device is provided according to a UGV scan pattern that isprovided locally by the UGV lidar device, wherein the UGV scan patternhas multiple scanning directions relative to the UGV lidar device.Similarly, the coordinative scan of the environment by the UAV lidardevice is provided according to a UAV scan pattern that is providedlocally by the UAV lidar device, wherein the UAV scan pattern hasmultiple scanning directions relative to the UAV lidar device.Furthermore, the UGV scan pattern provides the same local angulardistribution of the multiple scanning directions, the same angular pointresolution of its individual scanning directions, and the same distanceresolution as the UAV scan pattern. Thus, the UGV lidar data and the UAVlidar data are provided intrinsically with the same measurementparameters, which provides simplified referencing of the UGV lidar dataand the UAV lidar data to the common coordinate system.

By way of example, the UGV lidar device and the UAV lidar device are ineach case embodied as laser scanner, which is configured to generatelidar data by means of a rotation of a laser beam about two rotationaxes, wherein the laser scanner comprises a rotating body configured torotate about one of the two rotation axes and to provide for a variabledeflection of an outgoing and a returning part of the laser beam,thereby providing a rotation of the laser beam about the one of the tworotation axes, often referred to as fast axis. The rotating body isrotated about the fast axis with at least 50 Hz and the laser beam isrotated about the other of the two rotation axes, often referred to asslow axis, with at least 0.5 Hz. The laser beam is emitted as pulsedlaser beam, e.g. wherein the pulsed laser beam comprises 1.5 millionpulses per second, providing for a point acquisition rate of the lidardata of at least 300,000 points per second. For the rotation of thelaser beam about the two axes the field-of-view about the fast axis isat least 130 degrees and about the slow axis 360 degrees.

By way of example, the UGV lidar device/laser scanner is embodied thatwhen mounted on the UGV the slow axis is essentially arranged verticaland the UAV lidar device/laser scanner is embodied that when mounted onthe UAV the slow axis is essentially horizontal. Thus, the UAV lidardevice has real-time coverage of the area above the UAV, in front of theUAV, and the surface below the UAV. The UGV lidar device has real-timecoverage of the floor and the areas in front, above, and behind the UGV.

In some embodiments, the system comprises a UGV simultaneouslocalization and mapping unit, UGV SLAM unit, and a UAV simultaneouslocalization and mapping unit, UAV SLAM unit. The UGV SLAM unit isconfigured for reception of the UGV lidar data as UGV perception dataproviding a representation of the surroundings of the UGV lidar deviceat a current position, use of the UGV perception data to generate a UGVmap of an environment, and determination of a trajectory of a path thatthe UGV lidar device has passed within the UGV map of the environment.The UAV SLAM unit is configured for reception of the UAV lidar data asUAV perception data providing a representation of the surroundings ofthe UAV lidar device at a current position, use of the UAV perceptiondata to generate a UAV map of an environment, and determination of atrajectory of a path that the UAV lidar device has passed within the UAVmap of the environment.

In order to provide for sufficient data processing power, the system mayhave connection means for data exchange between the UGV lidar device andthe UAV lidar device with a data cloud which provides for cloudcomputing, e.g. to determine the 3D survey point cloud or to carry outat least part of the processing for the evaluation of a furthertrajectory of the UGV or the UAV, respectively. In particular on side ofthe UGV, the system can profit from on-board computing, e.g. by means ofa dedicated computing unit provided with the UGV lidar device or bymeans of a computing unit of the unmanned ground vehicle, whichsignificantly extends computing capabilities in case connection to thecloud is lost or in case data transfer rate is limited. Of course, thesame is possible for the UAV but typically with a UAV the load capacityand battery power are already limited. Another possibility is theinclusion of a connectivity to a companion device, e.g. a tablet, whichcould be configured to determine the 3D survey point cloud or to carryout at least part of the processing for the evaluation of the furthertrajectories of the UGV or the UAV similar than the cloud processing.The local companion device could then take over processing for areaswhere there is limited or no connectivity to the cloud, or the localcompanion device could serve as a cloud interface in the sense of arelay between on-board computing and cloud computing. By way of example,switching between on-board computing, cloud processing, and processingby the companion device is carried out dynamically as a function ofconnectivity between the three processing locations.

In some embodiments, the system comprises an on-board computing unitspecifically foreseen to be located on the unmanned ground vehicle andconfigured to carry out at least part of a system processing, whereinthe system processing comprises carrying out the SLAM process for theUGV or the UAV, providing the referencing of the UGV and UAV lidar data,and carrying out the evaluation of the further trajectories of eitherthe UGV and the UAV or both. The system further comprises an externalcomputing unit configured to carry out at least part of the systemprocessing. A communication module of the system is configured toprovide for a communication between the on-board computing unit and theexternal computing unit, wherein the system comprises a workloadselection module configured to monitor an available bandwidth of thecommunication module for the communication between the on-boardcomputing unit with the external computing unit, to monitor an availablepower of the on-board computing unit, the UGV and/or UAV lidar devices,the SLAM units of the UGV and UAV, and the path planning unit, and todynamically change an assignment of at least part of the systemprocessing to the on-board computing unit and the external computingunit depending on the available bandwidth and the available powerassigned to the external processing unit.

By way of example, localization is processed either locally on acomputing device (which is either part of the UGV or the UAV or which isa separate computing base station) or in the “cloud”. Similarly,computation with regard to marker detection and referencetransformation, sparse map generation, scan area definition and gapfilling may be distributed on different on-board, local, and cloud basedcomputing units.

For example, in order to minimize computation weight on the UAV, fastconnectivity between the UAV, the UGV, the companion device, and thecloud is implemented. For example, communication to the cloud is basedon a 4G/5G uplink, wherein a local connection (e.g. WLAN) is usedbetween UAV and UGV and/or companion device to download data from UAV tothe UGV and/or the companion device. On-board processing or companionprocessing is especially interesting if the UAV has a betterline-of-sight but bad cloud connectivity (e.g. when observing in acanyon).

In order to dynamically assign processing steps to the differentcomputing units, e.g. deciding where data is processed and how the datais uploaded, an arbitrator or scheduler unit (in the sense of astrategic controller) may be implemented on either the UGV, the UAV, thecompanion device or the base-station.

In some embodiments, the system is configured for carrying out a systemprocessing, which comprises carrying out a SLAM process associated withthe unmanned ground vehicle and/or the unmanned aerial vehicle,providing the referencing of the UGV lidar data and/or the UAV lidardata to the common coordinate system, and carrying out a path planningto determine a further trajectory to be followed by the unmanned groundvehicle and/or the unmanned aerial vehicle. The system comprises a UGVcomputing unit specifically foreseen to be located on the unmannedground vehicle and configured to carry out at least part of the systemprocessing. Similarly, the system comprises a UAV computing unitspecifically foreseen to be located on the unmanned aerial vehicle andconfigured to carry out at least part of the system processing and thesystem comprises an external computing unit configured to carry out atleast part of the system processing.

A communication unit of the system is configured to provide a mutualcommunication between the UGV computing unit, the UAV computing unit,and the external computing unit by using a cellular communicationconnection, e.g. 4G or 5G. The communication unit further provides for amutual communication between the UGV computing unit and the UAVcomputing unit by using a local communication connection, e.g. WLAN orBluetooth.

Here, the system further comprises a workload selection moduleconfigured to monitor an available bandwidth for the cellularcommunication connection and for the local communication connection tocarry out a dynamic change of an assignment of at least part of thesystem processing to the UGV computing unit, the UAV computing unit, andthe external computing unit. The dynamic change of the assignmentdepends on the available bandwidth for the cellular communicationconnection and for the local communication connection, wherein aprioritization rule is implemented to minimize the available processingload of the UAV computing unit before minimizing the availableprocessing load of the UGV computing unit, and to minimize the availableprocessing load of the UGV computing unit before minimizing theavailable processing load of the external computing unit.

The dynamic change of the assignment may further depend on theavailability of UAV and/or UGV battery power, i.e. wherein the workloadfor either the UAV and/or UGV computing unit is selected depending onwhich has most battery power available.

The dynamic change of the assignment may further depend on a requirementof SLAM associated with the UGV lidar data and/or UAV lidar data. Forexample, the dynamic change of the assignment may be based on arequirement to provide sufficient overlap of the sparse map with theknown digital model.

BRIEF DESCRIPTION OF THE DRAWINGS

The system according to the different aspects of the invention isdescribed or explained in more detail below, purely by way of example,with reference to working examples shown schematically in the drawing.Identical elements are labelled with the same reference numerals in thefigures. The described embodiments are generally not shown true to scaleand they are also not to be interpreted as limiting the invention.

FIG. 1 : an exemplary embodiment of an unmanned ground vehicle workingtogether with an unmanned aerial vehicle;

FIG. 2 : an exemplary embodiment of the lidar device of the unmannedground vehicle or the unmanned aerial vehicle, respectively;

FIG. 3 : exemplary embodiment of a reference unit comprising UGV markersand a UAV marker;

FIG. 4 : an exemplary workflow using a reference unit comprising UGVmarkers and a UAV marker, wherein the unmanned ground vehicle comprisesa further marker as reference for the unmanned aerial vehicle;

FIG. 5 : an exemplary embodiment of a marker, e.g. one of the UGVmarker, the UAV marker, and the further marker arranged on the UGV;

FIG. 6 : an exemplary communication scheme between the unmanned groundvehicle, the unmanned aerial vehicle, a companion device, and cloudprocessing;

FIG. 7 : further exemplary communication schemes with dynamic allocationof processing steps to different computing units.

DETAILED DESCRIPTION

FIG. 1 depicts an exemplary embodiment of an unmanned ground vehicle(UGV) 1, working together with an unmanned aerial vehicle (UAV) 2. Eachof the UGV 1 and the UAV 2 is equipped with a lidar device referred toas UGV lidar device 3 and UAV lidar device 4, respectively.

Here, the robotic ground vehicle 1 is embodied as a four-legged robot.For example, such robots are often used in unknown terrain withdifferent surface properties having debris and steep inclines. Theground robot 1 has sensors and processing capabilities to provide forsimultaneous localization and mapping, which comprises reception ofperception data providing a representation of the surroundings of theautonomous ground robot 1 at a current position, use of the perceptiondata to generate a map of the environment, and determination of atrajectory of a path that the ground robot 1 has passed within the mapof the environment.

The aerial vehicle 2 is embodied as quadcopter drone, which allowsfurther versatility to survey areas that are difficult or impossible toaccess by the robotic ground vehicle 1. Similarly to the UGV 1, theaerial vehicle 2 has sensors and processing capabilities to provide forsimultaneous localization and mapping, which comprises reception ofperception data providing a representation of the surroundings of theunmanned aerial vehicle 2 at a current position, use of the perceptiondata to generate a map of the environment, and determination of atrajectory of a path that the aerial vehicle 2 has passed within the mapof the environment.

Each of the UGV lidar device 3 and UAV lidar device 4 has afield-of-view of 360 degrees about a so-called slow axis 5 and aso-called band field-of-view 6 of at least 130 degrees about a fast axis(see FIG. 2 ). Both lidar devices 3, 4 are each configured to generatethe corresponding lidar data with a point acquisition rate of at least300′000 points per second. For example, the UGV lidar device 3 and theUAV lidar device 4 are each embodied as so-called two-axis laser scanner(see FIG. 2 ), wherein in case of the UGV lidar device 3, the fast axis5 is essentially aligned vertical and in case of the UAV lidar device 4,the fast axis 5 is essentially aligned horizontal.

The SLAM units of the UGV and the UAV respectively, are configured toreceive the corresponding lidar data as the perception data, which, forexample, provides improved field-of-view and viewing distance and thusimproved larger scale path determination. For example, this isparticularly beneficial for exploring unknown terrain. Another benefitcomes with the all-around horizontal field-of-view about the slow axis 5and the band field-of-view 6 of 130 degrees about the fast axis. In caseof the UGV 1 this provides the capability to essentially cover thefront, the back, and the ground at the same time, wherein in case of theUAV 2 this provides the capability to essentially cover the back and theground at the same time.

By way of example, the lidar data generated by means of the UGV lidardevice 3 and the UAV lidar device 4 can be combined for gap-filling ofcomplimentary system data. Typically, the UGV lidar device 3 “sees”objects close to the ground and in a side perspective (facades, soffit,etc.) and is used for indoor surveying (buildings, tunnels, etc.). TheUAV lidar device 4 observes objects above ground (upper level facades,roof, etc.) and is often used for outdoor surveying (buildings, bridges,etc.). In the figure, both the UAV lidar device 4 and the UGV lidardevice 3 are exemplarily used to coordinatively measure a pipe 7, e.g.on a power plant site, wherein the UAV lidar device 4 predominantlyobserves the top part of the pipe 7 and the UGV lidar device 3 onlyobserves the pipe 7 from a side perspective.

The combination of a UGV 1 and a UAV 2 further allows to carry out ascan area definition for the UGV 1 (or the UGV lidar device 3) by meansof an exploration flight of the UAV 2 and the UAV lidar device 4. By theexploration flight, a region of interest to be surveyed by the UGV lidardevice 3 is defined. For example, the UAV 2 provides for generation ofan overview of path, where UGV 1 is following. Spatial anchoring(re-localization) allows matching of the UGV lidar data and the UAVlidar data and trajectory alignment for line-of-sight environments.

The exploration by the UAV also allows to estimate if a particularmeasurement goal can be reached with constraints, e.g. providing for animproved estimate whether the battery of the UAV 2 or the UGV 1 isenough to fulfill a foreseen task. Since battery power of a UAV istypically limited, the UGV 1 may further be configured aslanding/docking station for the UAV 2 and as a moving charging stationfor the UAV 2. This way the reach by the UAV 2 can be extended byre-charging, e.g. during periods where only surveying by the UGV lidardevice 3 is required, e.g. when stepping in an indoor environment.Similarly, while heavy data download may be preferably carried out in adocked state of the UAV 2 on the UGV 1.

FIG. 2 shows an exemplary embodiment of the UGV lidar device 3 or theUAV lidar device 4, respectively, in the form of a so-called two-axislaser scanner. The laser scanner comprises a base 8 and a support 9, thesupport 9 being rotatably mounted on the base 8 about the slow axis 5.Often the rotation of the support 9 about the slow axis 5 is also calledazimuthal rotation, regardless of whether the laser scanner, or the slowaxis 5, is aligned exactly vertically.

The core of the laser scanner is an optical distance measuring unit 10arranged in the support 9 and configured to perform a distancemeasurement by emitting a pulsed laser beam 11, e.g. wherein the pulsedlaser beam comprises 1.5 million pulses per second, and by detectingreturning parts of the pulsed laser beam by means of a receiving unitcomprising a photosensitive sensor. Thus, a pulse echo is received froma backscattering surface point of the environment, wherein a distance tothe surface point can be derived based on the time of flight, the shape,and/or the phase of the emitted pulse.

The scanning movement of the laser beam 11 is carried out by rotatingthe support 9 relative to the base 8 about the slow axis 5 and by meansof a rotating body 12, which is rotatably mounted on the support 9 androtates about a so-called fast axis 14, here a horizontal axis. By wayof example, both the transmitted laser beam 11 and the returning partsof the laser beam are deflected by means of a reflecting surfaceintegral with the rotating body 12 or applied to the rotating body 12.Alternatively, the transmitted laser radiation is coming from the sidefacing away from the reflecting surface, i.e. coming from the inside ofthe rotating body 12, and emitted into the environment via a passagearea within the reflecting surface.

For the determination of the emission direction of the distancemeasuring beam 11 many different angle determining units are known inthe prior art. For example, the emission direction may be detected bymeans of angle encoders, which are configured for the acquisition ofangular data for the detection of absolute angular positions and/orrelative angular changes of the support 9 or of the rotating body 12,respectively. Another possibility is to determine the angular positionsof the support 9 or the rotating body 12, respectively, by onlydetecting full revolutions and using knowledge of the set rotationfrequency.

A visualization of the data can be based on commonly known dataprocessing steps and/or display options, e.g. wherein the acquired datais presented in the form of a 3D point cloud or wherein 3D vector filemodel is generated.

The laser scanner is configured to ensure a total field of view of themeasuring operation of the laser scanner of 360 degrees in an azimuthdirection defined by the rotation of the support 9 about the slow axis 5and at least 130 degrees in a declination direction defined by therotation of the rotating body 12 about the fast axis 14. In other words,regardless of the azimuth angle of the support 9 about the slow axis 5,the laser beam 11 can cover a so-called band field of view, in thefigure a vertical field of view, spread in the declination directionwith a spread angle of at least 130 degrees.

By way of example, the total field of view typically refers to a centralreference point 13 of the laser scanner defined by the intersection ofthe slow axis 5 with the fast axis 14.

FIG. 3 exemplary shows an embodiment of a reference unit 15 according tothe invention, comprising UGV markers 16A, 16B and a UAV marker 17.

Here, the reference unit 15 is embodied in the shape of a cube. In anominal setup, e.g. wherein one of the sides of the cube is alignedexactly horizontally, the reference unit 15 provides four (vertical)sides useable to provide for UGV markers 16A, 16B and one (horizontal)side useable to provide for a UAV marker 17. For example, the setting upof the cube in its nominal setup is aided by a bubble level.

Here, the UGV markers 16A, 16B and the UAV marker 17 comprise visiblecodes providing spatial 3D information on the spatially fixedarrangement of the UGV markers 16A, 16B and the UAV marker 17 withrespect to each other. The spatial 3D information can be determined byreading the codes by visual pickup units arranged on the UGV and theUAV, respectively, e.g. cameras or the UGV and UAV lidar devices.

When the UGV and the UGV lidar device 3 pass the reference cube 15, avisible UGV marker 16B is automatically identified and a coordinativemeasurement of the visible UGV marker 16B is carried out by the UGVlidar device 3, whereby relative position data providing relativeposition information of the visible UGV marker 16B with respect to theUGV lidar device 3 is determined. Thus, the relative position andparticularly orientation of the moving UGV lidar device 3 with respectto the identified visible UGV marker 16B is tracked such that it can beused to facilitate later detection of the UAV marker 17 by the, e.g.starting, UAV.

By way of example, upon start of the UAV, the relative position data andthe determined spatial 3D information on the spatially fixed arrangementof the identified visible UGV marker 16B and the UAV marker 17 withrespect to each other are taken into account to carry out an automaticdetection of the UAV marker 17 and to carry out a coordinativemeasurement of the UAV marker 17 by the UAV lidar device 4. Thecoordinative measurement of the identified visible UGV marker 16B andthe coordinative measurement of the UAV marker 17 are then taken intoaccount to provide the referencing of the UGV lidar data and the UAVlidar data with respect to a common coordinate system.

FIG. 4 depicts a further exemplary workflow using a reference unit 15comprising UGV markers and a UAV marker, wherein the unmanned groundvehicle comprises a further marker 18 as reference for combining the UAVlidar data and the UGV lidar data in the common coordinate system.

The further marker 18 is arranged on the UGV and used to provide a linkbetween the UAV lidar device 4 and the UGV lidar device 3 along themobile mapping process. The UAV lidar device 4 is configured toautomatically carry out a coordinative measurement of the further marker18 in order to take into account the coordinative measurement of thefurther marker 18 to provide the referencing of the UGV lidar data andthe UAV lidar data with respect to the common coordinate system. Forexample, in case positional relationships of different reference unitswithin the environment are known, e.g. absolute positions of differentreference units given in an outer coordinate system, a larger baselineto overcome inaccuracies in the coordinative measurements of thereferenced start marker is provided and, for example, can be used forso-called loop closure of the SLAM algorithms, which allows tocompensate positional drifts when referencing (“stitching together”)lidar data of different positions along the way of the UGV or the AGV.

Some of the markers, e.g. one of the UGV marker 16A, 16B, the UAV marker17, and the further marker 18 arranged on the UGV, may further comprisea reference value indication, which provides positional information,e.g. 3D coordinates, regarding a set pose of the marker in the commoncoordinate system or in an outer coordinate system, e.g. aworld-coordinate system. The set pose is a 6DoF pose, i.e. position andorientation of the marker, and indicates the desired 6DoF pose of themarker. Thus, when correctly placed in the environment, this marker canact as so-called survey control point, e.g. for loop closure of a SLAMprocess and/or as absolute reference in a world coordinate system or alocal site coordinate system.

By way of example, the system is configured to derive the set pose andto take into account the set pose for the referencing of the UGV lidardata and the UAV lidar data in the common coordinate system, e.g. bydetermining a pose of the marker in the common coordinate system or inthe world coordinate system and carrying out a comparison of thedetermined pose of the marker and the set pose.

FIG. 5 depicts an exemplary embodiment of a marker 30, e.g. one of theUGV marker 16A, 16B, the UAV marker 17, and the further marker 18arranged on the UGV (see FIGS. 3 and 4 ). On the left, the marker 30 isshown in a frontal view. On the right, the marker 30 is shown in anangled view.

The marker comprises a visually detectable pattern, e.g. provided byareas of different reflectivity, different gray scales and/or differentcolors. The pattern comprises a circular feature 31 and inner geometricfeatures 32, which are enclosed by the circular feature 31.

By way of example, the system is configured to determine the 6DoF (sixdegrees of freedom) pose of the marker. The 6DoF pose is derived bydetermining a 3D orientation of the pattern, i.e. a 3D orientation of apattern plane, and by determining a 3D position of the pattern. Forexample, marker corners 33 (at least three) are analyzed to provide fordetermination of an angle of the pattern plane. The marker corners 33may be determined using a camera on the UGV or the UAV, respectively.

The circular feature 31 provides for improved determination of the 3Dorientation of the pattern plane. By way of example, the system isconfigured to generate an intensity image of the pattern by a scanningof the pattern with a lidar measurement beam of the UGV lidar device orthe UAV lidar device, respectively, wherein the intensity image isgenerated by detection of an intensity of a returning lidar measurementbeam. By identifying the image of the circular feature within theintensity image of the pattern and running a plane fit algorithm to fitan ellipse to the image of the circular feature the 3D orientation ofthe pattern plane is determined with improved precision. In addition,the center of the ellipse may be determined and used as aiming point forthe lidar device to determine the 3D position of the pattern, therebyallowing to determine the 6DoF pose of the pattern.

The 3D orientation of the pattern, particularly the 6DoF pose of thepattern, are then taken into account for providing the referencing ofthe UGV lidar data and the UAV lidar data with respect to the commoncoordinate system.

FIG. 6 depicts an exemplary communication scheme between the unmannedground vehicle, e.g. the UGV lidar device 3, the unmanned aerialvehicle, e.g. the UAV lidar device 4, a companion device 19, e.g. atablet, and a data cloud 20 providing for cloud processing.

By way of example, an operator's tablet 19 is locally connected, e.g. bymeans of a Bluetooth or WLAN connection to the UAV lidar device 4 andthe UGV lidar device 3, wherein the tablet allows mediating control ofboth lidar devices 3, 4. The tablet 19 is further connected to a cloudprocessing unit 20.

An optional connection between UAV lidar device 4 and UGV lidar device 3provides redundancy in case connectivity to the tablet 19 is lost. Cloudconnectivity of the tablet 19, the UAV lidar device 4, and the UGV lidardevice 3 allows operation without local connection and provides anadditional fallback scenario. For example, cloud connectivity isestablished via 4G/5G uplink.

For example, such versatile communication capability allows to implementdynamic distribution of processing and data storage, e.g. to coordinatea desired data processing rate and battery life.

FIG. 7 shows exemplary communication schemes with dynamic allocation ofprocessing steps to different computing units. Here, the unmanned groundvehicle comprises an on-board computing unit 21 and a cellularcommunication uplink 22 to the cloud 20. Similarly, the UAV comprises acellular communication uplink (not shown) to the cloud 20. In two bottomschematics, the system further comprises a base-station 23 located closethe UGV and configured for comparatively heavy computing (compared tothe on-board computing unit 21). The base-station 23 may also have acellular communication uplink 22 to the cloud 20 (bottom left schematic)or data upload to the cloud 20 may predominantly be carried out over theUGV cellular communication uplink 22 (bottom right schematic), e.g.wherein the UGV uplink 22 acts as relay between the base station 23 andthe cloud 20.

The on-board computing unit 21 of the UGV and the base-station 23provide to minimize processing on the UAV and thus to save battery lifeof the UAV.

In the top left schematic a local data connection, e.g. by WLAN, isestablished between the UAV lidar device 4 and the on-board computingunit 21 of the UGV in order to download data from the UAV lidar device4. The UGV on-board computing unit, which has more payload capability,is computing results in-field, and provides an uplink functionality tocloud computing services 20.

In the top right schematic, a local connection, e.g. by WLAN, isestablished to upload data from the UGV on-board computing unit 21and/or the UGV lidar device to the UAV cellular uplink (not shown),which provides the data to the cloud 20. For example, this approach isused if the UAV has a better line-of-sight or connectivity to the cloud20, e.g. when the UGV is walking in a canyon with limited or noconnectivity.

In the bottom left schematic, local connections are established in orderto download data from the UAV lidar device 4, the UGV lidar device 3,and the UGV on-board computing unit 21 to the base station 23. The mainprocessing payload is on side of the base-station 23 and the cloud 20,which have established a cellular data connection between each other.

Similarly, in the bottom right schematic main processing is on side ofthe base-station 23 and the cloud 20, but communication with the cloud20 is routed over the UGV data uplink 22.

By way of example, an arbitrator or scheduler unit, e.g. located on theUGV or on the base station 23 is used to dynamically distributeprocessing to the different processing units, e.g. to distribute atleast parts of calculating further trajectories, calculating maps of theSLAM process, and referencing the UGV lidar data and the UAV lidar datato a common coordinate system. The arbitratror or scheduler unit mayalso define where and how the data is uploaded/downloaded to/from thecloud 20.

In particular, switching between on-board computing, cloud processing,processing by the lidar devices, and processing by the companion deviceis carried out dynamically as a function of connectivity between thecomputing locations and available power on the UGV and the UAV.Typically, whenever possible processing is taken away from the UAV, andpossibly also from the UGV, e.g. to the cloud, the companion device, andthe base-station, because battery power and data storage of the UAV andUGV (and the devices located on the UAV and UGV) are limited.

Although the invention is illustrated above, partly with reference tosome preferred embodiments, it must be understood that numerousmodifications and combinations of different features of the embodimentscan be made. All of these modifications lie within the scope of theappended claims.

1. A system for providing 3D surveying of an environment, wherein thesystem comprises a first and a second lidar device, wherein: one of thefirst and the second lidar device, is an unmanned ground vehicle (UGV)lidar device configured to be mounted on an unmanned ground vehicle andconfigured to generate UGV lidar data to provide a coordinative scan ofthe environment relative to the UGV lidar device, the other of the firstand the second lidar device is an unmanned aerial vehicle (UAV) lidardevice configured to be mounted on an unmanned aerial vehicle andconfigured to generate UAV lidar data to provide a coordinative scan ofthe environment relative to the UAV lidar device, and the system isconfigured to provide a referencing of the UGV lidar data and the UAVlidar data with respect to a common coordinate system for determining a3D survey point cloud of the environment, wherein: the system comprisesa reference unit comprising a first and a second marker, wherein thefirst and the second marker are in a spatially fixed arrangement withrespect to each other and each of the first and the second marker isconfigured as target for a coordinative measurement of the respectivemarker by a lidar device, wherein the system is configured: to carry outan automatic detection of the first marker and to carry out acoordinative measurement of the first marker by the first lidar deviceto determine relative position data providing relative positioninformation of the first marker with respect to the first lidar device,to take into account the relative position data and a spatial 3Dinformation on the spatially fixed arrangement of the first and thesecond marker with respect to each other to carry out an automaticdetection of the second marker and to carry out a coordinativemeasurement of the second marker by the second lidar device, and to takeinto account the coordinative measurement of the first marker and thecoordinative measurement of the second marker to provide the referencingof the UGV lidar data and the UAV lidar data with respect to the commoncoordinate system.
 2. The system according to claim 1, wherein: one ofthe first marker and the second marker is an UGV marker that isconfigured that in a nominal setup of the reference unit it is spatiallyarranged in such a way that the UGV lidar device can carry out acoordinative measurement of the UGV marker, wherein the coordinativemeasurement of the UGV marker is carried out from a sideways lookingfield-of-view associated with the montage of the UGV lidar device on theUGV, the other of the first and the second marker is an UAV marker thatis configured that in the nominal setup of the reference unit, it isspatially arranged in such a way that the UAV lidar device can carry outa coordinative measurement of the UAV marker, wherein the coordinativemeasurement of the UAV marker is carried out from a downward lookingfield-of-view associated with the montage of the UAV lidar device on theUAV.
 3. The system according to claim 1, wherein: the system isconfigured to access assignment data, which provide the spatial 3Dinformation on the spatially fixed arrangement of the first and thesecond marker with respect to each other, and/or at least one of thefirst marker and the second marker comprises a visible code, optionallya barcode, or optionally a matrix barcode, which provides the spatial 3Dinformation on the spatially fixed arrangement of the first and thesecond marker with respect to each other, wherein the system isconfigured to determine the spatial 3D information on the spatiallyfixed arrangement of the first and the second marker with respect toeach other by using a visual pick-up device.
 4. The system according toclaim 1, wherein: the coordinative scan of the environment by the UGVlidar device is provided according to a UGV scan pattern that isprovided locally by the UGV lidar device, wherein the UGV scan patternhas multiple scanning directions relative to the UGV lidar device, thecoordinative scan of the environment by the UAV lidar device is providedaccording to a UAV scan pattern that is provided locally by the UAVlidar device, wherein the UAV scan pattern has multiple scanningdirections relative to the UAV lidar device, and the UGV scan patternprovides the same local angular distribution of the multiple scanningdirections, the same angular point resolution of its individual scanningdirections, and the same distance resolution as the UAV scan pattern. 5.The system according to claim 1, wherein: the UGV lidar device and theUAV lidar device are in each case embodied as a laser scanner, which isconfigured to generate lidar data by means of a rotation of a laser beamabout two rotation axes, wherein: the laser scanner comprises a rotatingbody configured to rotate about one of the two rotation axes and toprovide for a variable deflection of an outgoing and a returning part ofthe laser beam, thereby providing a rotation of the laser beam about theone of the two rotation axes, fast axis, the rotating body is rotatedabout the fast axis with at least 50 Hz, the laser beam is rotated aboutthe other of the two rotation axes, slow axis, with at least 0.5 Hz, thelaser beam is emitted as pulsed laser beam, particularly wherein thepulsed laser beam comprises 1.5 million pulses per second, providing fora point acquisition rate of the lidar data of at least 300,000 pointsper second, and for the rotation of the laser beam about the two axesthe field-of-view about the fast axis is 130 degrees and about the slowaxis 360 degrees.
 6. The system according to claim 1, wherein: the firstand the second marker are arranged on a common component such that therelative spatial arrangement of the first and the second marker ismechanically fixed, optionally, wherein the common component comprisesan alignment indicator providing for a visual determination of analignment of the common component with respect to an outer coordinatesystem or with respect to a cardinal direction to establish the nominalsetup.
 7. The system according to claim 1, wherein: at least one of thefirst and the second marker comprises a visually detectable pattern,optionally provided by areas of different reflectivity, different grayscales and/or different colors, the system is configured to determine a3D orientation of the pattern by: determining geometric features in anintensity image of the pattern, wherein the intensity image of thepattern is acquired by a scanning of the pattern with a lidarmeasurement beam of the UGV lidar device or the UAV lidar device and adetection of an intensity of a returning lidar measurement beam, andcarrying out a plane fit algorithm in order to determine an orientationof a pattern plane, by analyzing an appearance of the geometric featuresin the intensity image of the pattern, and the system is configured totake into account the 3D orientation of the pattern for providing thereferencing of the UGV lidar data and the UAV lidar data with respect tothe common coordinate system.
 8. The system according to claim 7,wherein: the pattern comprises a circular feature, the system isconfigured to identify an image of the circular feature within theintensity image of the pattern, and the plane fit algorithm isconfigured to fit an ellipse to the image of the circular feature and,based thereof, to determine the orientation of the pattern plane,particularly wherein the center of the ellipse is determined and aiminginformation for aiming with the lidar measurement beam to the center ofthe ellipse are derived, optionally, wherein the pattern comprises innergeometric features, particularly comprising rectangular features, whichare enclosed by the circular feature.
 9. The system according to claim1, wherein the first and the second marker each comprise a visibleindication of the directions of at least two of the three main axeswhich span the common coordinate system, wherein the system isconfigured to determine the directions of the three main axes by usingthe UGV lidar device and the UAV lidar device, and to take into accountthe directions of the three main axes for providing the referencing ofthe UGV lidar data and the UAV lidar data with respect to the commoncoordinate system.
 10. The system according to claim 1, wherein thesystem is configured that the coordinative measurement of the firstmarker is carried out by the UGV lidar device and the coordinativemeasurement of the second marker is carried out by the UAV lidar device,wherein the automatic detection of the second marker and thecoordinative measurement of the second marker by the UAV lidar device iscarried out at each take-off and landing of the unmanned aerial vehicle,optionally, wherein the system is configured that the relative positiondata is continuously updated so that the relative position informationprovides continuously updated spatial information about the arrangementbetween the first marker and the UGV lidar device.
 11. The systemaccording to claim 1, wherein: the system comprises a further marker, inaddition to the first and the second marker, which is configured to bemounted on the unmanned ground vehicle, the UAV lidar device isconfigured to automatically carry out a coordinative measurement of thefurther marker, and the system is configured to take into account thecoordinative measurement of the further marker to provide thereferencing of the UGV lidar data and the UAV lidar data with respect tothe common coordinate system.
 12. The system according to claim 1,wherein: the system comprises a visual pick-up device configured to bearranged on the unmanned ground vehicle or the unmanned aerial vehicle,optionally, wherein the visual pick-up device is a camera or one of theUGV lidar device or the UAV lidar device, the system is configured togenerate a sparse map using the visual pick-up device and to carry out alocalization of the UGV lidar data or the UAV lidar data in the sparsemap.
 13. The system according to claim 12, wherein: the sparse map isgenerated by photogrammetric triangulation and the localizationcomprises a first referencing between the UGV lidar data and the UAVlidar data, and after the first referencing, a second referencingbetween the UGV lidar data and the UAV lidar data is carried out basedon point-cloud matching between the UGV lidar data and the UAV lidardata, wherein the sparse map is referenced with respect to a knowndigital model of the environment.
 14. The system according to claim 1,wherein the system comprises a UGV simultaneous localization and mappingunit (UGV SLAM unit) and a UAV simultaneous localization and mappingunit, UAV SLAM unit, wherein: the UGV SLAM unit is configured forreception of the UGV lidar data as UGV perception data providing arepresentation of the surroundings of the UGV lidar device at a currentposition, use of the UGV perception data to generate a UGV map of anenvironment, and determination of a trajectory of a path that the UGVlidar device has passed within the UGV map of the environment, and theUAV SLAM unit is configured for reception of the UAV lidar data as UAVperception data providing a representation of the surroundings of theUAV lidar device at a current position, use of the UAV perception datato generate a UAV map of an environment, and determination of atrajectory of a path that the UAV lidar device has passed within the UAVmap of the environment.
 15. The system according to claim 1, wherein thesystem is configured for carrying out a system processing, whichcomprises carrying out a SLAM process associated with the unmannedground vehicle and/or the unmanned aerial vehicle, providing thereferencing of the UGV lidar data and/or the UAV lidar data to thecommon coordinate system, and carrying out a path planning to determinea further trajectory to be followed by the unmanned ground vehicleand/or the unmanned aerial vehicle, wherein the system comprises: a UGVcomputing unit configured to be located on the unmanned ground vehicleand configured to carry out at least part of the system processing, aUAV computing unit configured to be located on the unmanned aerialvehicle and configured to carry out at least part of the systemprocessing, an external computing unit configured to carry out at leastpart of the system processing, a communication unit configured: toprovide a mutual communication between the UGV computing unit, the UAVcomputing unit, and the external computing unit by using a cellularcommunication connection, particularly 4G or 5G, and to provide a mutualcommunication between the UGV computing unit and the UAV computing unitby using a local communication connection, particularly WLAN orBluetooth, a workload selection module configured to monitor anavailable bandwidth for the cellular communication connection and forthe local communication connection to carry out a dynamic change of anassignment of at least part of the system processing to the UGVcomputing unit, the UAV computing unit, and the external computing unit,wherein the dynamic change of the assignment depends: on the availablebandwidth for the cellular communication connection and for the localcommunication connection, and a prioritization rule to minimize theavailable processing load of the UAV computing unit before minimizingthe available processing load of the UGV computing unit and to minimizethe available processing load of the UGV computing unit beforeminimizing the available processing load of the external computing unit.